Complex Distributed Systems:The Need for Fresh Perspectives

Distributed systems are at a watershed due to their increasing complexity. The heart of the problem is the extreme level of heterogeneity exhibited by contemporary distributed systems coupled with the need to be dynamic and responsive to change. In effect, we have moved from distributed systems to systems of systems. Following on from this, middleware is also at a watershed. The traditional view of middleware is no longer valid (i.e. as a layer of abstraction, masking the complexity of the underlying distributed system and providing a high-level programming model). In practice, middleware is often by-passed with complex systems constructed in a rather ad hoc manner as a mash-up of a variety of technologies. The end result is that middleware is no longer sure of its form or purpose and this lack of a viable approach is a huge barrier to the emergence of areas such as smart cities and emergency response systems. This paper argues that there is a need to fundamentally rethink the middleware landscape related to complex distributed systems. The core contribution of the paper is a set of fresh perspectives, which lead us in turn to novel principles and patterns for middleware and subsequently to new styles of platform. These perspectives include a move to emergent middleware, seeking flexible meta-structures for distributed systems, and a step away from generic to domain-specific technologies. A number of case studies are also presented to demonstrate what this might mean for future distributed systems

Daleel:simplifying cloud instance selection using machine learning

Decision making in cloud environments is quite challenging due to the diversity in service offerings and pricing models, especially considering that the cloud market is an incredibly fast moving one. In addition, there are no hard and fast rules; each customer has a specific set of constraints (e.g. budget) and application requirements (e.g. minimum computational resources). Machine learning can help address some of the complicated decisions by carrying out customer-specific analytics to determine the most suitable instance type(s) and the most opportune time for starting or migrating instances. We employ machine learning techniques to develop an adaptive deployment policy, providing an optimal match between the customer demands and the available cloud service offerings. We provide an experimental study based on extensive set of job executions over a major public cloud infrastructure

Evaluation of the dynamic construct competition miner for an eHealth system

Business processes of some domains are highly dynamic and increasingly complex due to their dependencies on a multitude of services provided by various providers. The quality of services directly impacts the business process’s efficiency. A first prerequisite for any optimization initiative requires a better understanding of the deployed business processes. However, the business processes are either not documented at all or are only poorly documented. Since the actual behaviour of the business processes and underlying services can change over time it is required to detect the dynamically changing behaviour in order to carry out correct analyses. This paper presents and evaluates the integration of the Dynamic Construct Competition Miner (DCCM) as process monitor in the TIMBUS architecture. The DCCM discovers business processes and recognizes changes directly from an event stream at run-time. The evaluation is carried out in the context of an industrial use-case from the eHealth domain. We will describe the key aspects of the use-case and the DCCM as well as present the relevant evaluation results

A Framework for SLO-driven Cloud Specification and Brokerage

The diversity of cloud offerings motivated the proposition of cloud modelling languages (CMLs) to abstract complexities related to selection of cloud services. However, current CMLs lack the support for modelling service level objectives (SLOs) that are required for the customer applications. Consequently, we propose an application- and provider-independent SLO modelling language (SLO-ML) to enable customers to specify the required SLOs. We also sketch the architecture to realise SLO-ML

Advancing reproducible research by publishing R markdown notebooks as interactive sandboxes using the learnr package

Various R packages and best practices have played a pivotal role to promote the Findability, Accessibility, Interoperability, and Reuse (FAIR) principles of open science. For example, (1) well-documented R scripts and notebooks with rich narratives are deposited at a trusted data centre, (2) R Markdown interactive notebooks can be run on-demand as a web service, and (3) R Shiny web apps provide nice user interfaces to explore research outputs. However, notebooks require users to go through the entire analysis, while Shiny apps do not expose the underlying code and require extra work for UI design. We propose using the learnr package to expose certain code chunks in R Markdown so that users can readily experiment with them in guided, editable, isolated, executable, and resettable code sandboxes. Our approach does not replace the existing use of notebooks and Shiny apps, but it adds another level of abstraction between them to promote reproducible science

SE in ES:Opportunities for Software Engineering and Cloud Computing in Environmental Science

New and emergent computing architectures and software engineering practices provide an opportunity for environmental models to be deployed more efficiently and democratically. In this paper we aim to capture the software engineering practices of environmental scientists, highlight opportunities for software engineering and work towards developing a domain specific language for the configuration and deployment of environmental models. We hold a series of interviews with environmental scientists involved in developing and deploying computer based environmental models about the approach taken in engineering models, and describe a case study in deploying an environmental model (WRF: Weather Research & Forecasting) on a cloud architecture. From these studies we find a number of opportunities for a) software engineering methods and tools such as Domain Specific Languages to play a role in abstracting from underlying computing complexity, and for b) new architectures to increase efficiency and availability of deployment. Together, we propose they will allow scientists to concentrate on fundamental science rather than specifics of the underlying computing

The design and deployment of an end-to-end IoT infrastructure for the natural environment

Internet of Things (IoT) systems have seen recent growth in popularity for city and home environments. We report on the design, deployment, and use of the IoT infrastructure for environmental monitoring and management. Working closely with hydrologists, soil scientists, and animal behaviour scientists, we successfully deployed and utilised a system to deliver integrated information across these two fields in the first such example of real-time multidimensional environmental science. We describe the design of this system; its requirements and operational effectiveness for hydrological, soil, and ethological scientists; and our experiences from building, maintaining, and using the deployment at a remote site in difficult conditions. Based on this experience, we discuss key future work for the IoT community when working in these kinds of environmental deployments

A Spitzer Space Telescope far-infrared spectral atlas of compact sources in the Magellanic Clouds. I. The Large Magellanic Cloud

[abridged] We present 52-93 micron spectra obtained with Spitzer in the MIPS-SED mode, of a representative sample of luminous compact far-IR sources in the LMC. These include carbon stars, OH/IR AGB stars, post-AGB objects and PNe, RCrB-type star HV2671, OH/IR red supergiants WOHG064 and IRAS05280-6910, B[e] stars IRAS04530-6916, R66 and R126, Wolf-Rayet star Brey3a, Luminous Blue Variable R71, supernova remnant N49, a large number of young stellar objects, compact HII regions and molecular cores, and a background galaxy (z~0.175). We use the spectra to constrain the presence and temperature of cold dust and the excitation conditions and shocks within the neutral and ionized gas, in the circumstellar environments and interfaces with the surrounding ISM. Evolved stars, including LBV R71, lack cold dust except in some cases where we argue that this is swept-up ISM. This leads to an estimate of the duration of the prolific dust-producing phase ("superwind") of several thousand years for both RSGs and massive AGB stars, with a similar fractional mass loss experienced despite the different masses. We tentatively detect line emission from neutral oxygen in the extreme RSG WOHG064, with implications for the wind driving. In N49, the shock between the supernova ejecta and ISM is revealed by its strong [OI] 63-micron emission and possibly water vapour; we estimate that 0.2 Msun of ISM dust was swept up. Some of the compact HII regions display pronounced [OIII] 88-micron emission. The efficiency of photo-electric heating in the interfaces of ionized gas and molecular clouds is estimated at 0.1-0.3%. We confirm earlier indications of a low nitrogen content in the LMC. Evidence for solid state emission features is found in both young and evolved object; some of the YSOs are found to contain crystalline water ice.Comment: Accepted for publication in The Astronomical Journal. This paper accompanies the Summer 2009 SAGE-Spec release of 48 MIPS-SED spectra, but uses improved spectrum extraction. (Fig. 2 reduced resolution because of arXiv limit.